This is a command line method of maintaining your sanity when dealing with tapes. The TarChive package is a set of Perl scripts, and two binary files that together make a sensible way to deal with tapes.
This is a preliminary release. (escape?) Use with the due caution for any beta software. I appreciate bugs reports, suggestions, etc.
Tapes have several drawbacks as means of storing information.
They're slow.
You have to read the whole tape to find out what's on it.
You often aren't quite sure if what you think is there is really there.
You have little protection from someone else writing on the tape if you leave it in the drive.
If you have serveral tapes they are difficult for programs to tell apart. (Generally you have to go look at the label.)
If there is a glitch in the tape, you've usually lost the rest of the archive.
TarChive has partial solutions to most of the problems above.
Speed: TarChive encourages creating archives in the 20-100 MB range. Because of this you can use the much faster file positioning commands (10-100 times faster than reading the tape).
Directories: TarChive keeps log files of whatever it puts on tape. So later, a single grep command will tell you that file x is in the 11th archive on tape Y.
Verification: TarChive writes an archive to tape, then reads it back and compares it to the original.
Tape Labeling: TarChive will not write on a tape that is not labeled. TarChive will not write on a tape not owned by the user. TarChive will not write on a tape that is not registered on the system. All log files refer to the tape label so that you can easily determine what is on each tape.
Error recovery: TarChive uses gnutar as it's tarring command. If gnutar can't read a part of the tape, it moves on to the next file header within the archive, and carries on from there. Losses are minimized.
Driver glitches: On occasion Next's tape driver decides to reset the tape. TarChive keeps track of what tapefile is being written, and when a reset occures, repositions the tape to the correct tar file, and carries on.
Prerequisites:
In addition to this package, you need gnutar 1.10 and perl 4.010
Manifest:
mtsense & mtsense.c
A hack to find out what the drive is doing, and to set it's block size and mode. Credits are in the source.
TapeLibrary.perl
This is the guts of TarChive. A set of routines that perform various tape operations.
TapeLabel
A perl script to write a label on a tape and register that label on the host machine. TapeLabel checks for an existing label in the register, and checks for a label or data on the tape itself.
TapeErase
A perl script to remove a label from tape. The user must specify the label to be erased. This is checked in the register to see that the tape is registered. The user is not permitted to erase the tape unless he owns it, and the label given on the command line matches the one on the tape. Under these circumstances the label is unregistered, log files are deleted, and a new End of Data mark put at the beginning of the tape.
UserBack
A perl script that takes a list of directories as arguements, and creates a tar file on tape for each directory. UserBack is smart enough to count how many files are already on the tape, and move the tape appropriatly. Log files are put in users ~/backups.
FullUserBack
A perl script that pulls the list of users out of either netinfo or yellow pages, and creates a tar file for each user. The log file is put in the users ~/backups directory.
FUBcron
A csh script suitable for running from your crontab file. FUBcron
runs FullUserBack and mails the results to root.
Installation:
Create a directory for tape labels. I suggest /usr/local/tapelabels. This directory should be mounted where all machines that have a tape drive can read it. The directory should be owned by root, but have permissions rwxrwxrwt.
mkdir /usr/local/tapelabels
chmod a+rwx,+t /usr/local/tapelabels
This allows users to label and erase their own tapes, but not others.
Copy TapeLibrary.perl to wherever you keep perl libraries. /usr/local/lib/perl by default.
Copy TapeErase TapeLabel UserBack to a place on user's paths.
usually /usr/local/bin or /usr/local/scripts
Edit TapeLibrary.perl, and customize as appropriate. (Look for the word customize) You will at least have to tell it which hosts have exabyte drives.
Edit FullUserBack and customize as appropriate.
You will have to tell it what filesystems have users on them.
Put mtsense in a directory with general access. /usr/local/bin
Overview: Theory of operation:
When a tape is registered, a zero length file is created in /usr/local/tapelables. This file is owned by whoever ran TapeLabel, and is later used to determine who owns the tape. The name of this file is the name of the tape. The user must own the tape in order to write to it.
This zero length file is placed as a tar file at the beginning of the tape, and is read before any operations are done to the tape.
Each user has a backups subdirectory under his home directory. FullUserBack creates one if necessary, UserBack will fuss if it doesn't exist. This directory is normally owned by root so that users cannot erase files from it. They should be able to read it. You may wish to allow users to edit the files.
When material is backed up it is always done from the parent directory of the one requested. This way, on a restore, it is always restored to a single subdirectory underneath the directory where the restore was run. E.g. If I ask for a backup of ~/myprojects/TarChive it is stored as ./TarChive... Later if I restore it, while in ~/proj, it will be restored as ~/proj/TarChive...
When material is verified, the log of the verification pass is put into the users backups directory. The log file name is of the form:
TapeLabel-directory-date.
Inside, the header of the file looks like this:
Full User Backup: File 35 on Tape Gimel-DAT
Directory: /u/fenris/sherwood
Directory stored as ./sherwood
Date of backup: Tue Aug 18 13:56:39 MDT 1992
Archive size: 210310 /u/fenris/sherwood
This file will be deleted when tape is erased.
Gimel-DAT-sherwood-18-Aug-92 Volume 1
sherwood/
sherwood/.Active.datebk
...
This header is useful when you have to recover information from the tape.
Potentially there is a problem with the nameing convention used. If a user creates two backups of the directories with the same basename on the same tape on the same day their log files will have the same name. This causes the match between log files and files on tape to be lost. Example: ~/myprojects/TarChive/C-source and ~/myprojects/LaBrea/C-source both have base names of C-source. If I made backups on the same day on the same tape they would both have log file names of Sherwood_Data-C-Source-19Aug92. At present UserBack doesn't check for this.
The filenumber in the header is actually off by 1, as it doesn't count the tapelabel, which is really just a file. However, it's convenient this way, because it is the number of fsf's you need to do to get to the start of this file.
At present there is no set of recovery scripts. If I wished to recover data from the file above, I would do this:
setenv TAPE /dev/nrst0
mt fsf 35
gnutar -xvb 16 "filespec1 filespec2 filespec3..."
The v gives me a verbose list, and mentions the volume name of the archive. b 16 is block to sixteen 512 byte blocks per tape block. (Gnutar doesn't always pick this up when reading DAT drives.)
The filespec's can be wildcards. Remember however anything that matches a directory recovers the entire directory subtree.
Note: I've had troubles on occasion with large fsf's on exabyte drives. Seems that the driver times out waiting for the drive to say, "I'm done". If this is a problem, break it up in to smaller chunks. The tapelibrary does them 1 at a time in a loop to avoid this.
These routines were originally written for an exabyte 8200, then ported to and HP-DAT. (minimal changes) I have not tried this with a QIC drive yet.
Send comments, bugs, flames, etc to
sherwood@space.ualberta.ca
require "Disclaimer.perl"
I do not guarantee that this works.
I do not accept responsiblity for anything it does to your system.
